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ABSTRACT 


This thesis compared the use of a discrete utterance 
mouce —sTScogenition system and a keyboard@émtry device in 
retargeting Air Launch Cruise Missiles (ALCM) prior to 
munen trom a EB=ce2eG aircraft. Pime™ to 1¢0ad, input dnd 
cutput accuracies, and time versus accuracy measurements 
were made for each Of twenty subjects. Aeyboard entry was 
meund to be tetter than voice Entry in time to load and 
Peut acecurecy . These findings are limrited to discrete 
ieeerence VOlce MScoznition systems ahd most fprobabiy would 
have been different if a connected speech recognition system 
hed peen utilized. 

Also investigated were three display formats HO 
presenting flex targeting information on a Cathcede ray tube. 
information was updated on the cathode ray tube using 
Single-space, dcuble-space, and inverse-video formats. Time 
memupaave, input and output accuracies, and time versus 
accuracy measurements were meCOrded. NO- Wseleni? ean t 


differences were found among the three display formats. 
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I. BACKGROUND 


A. INTRODUCTION 

The E-52G was selected in 1977 to be outfitted with an 
all new navigation and weapons control suite called the 
Cttensive Avionics System (CAS). This system would control 
omc letneh theeair Launch Cruise. Missile (ALCM). The flight 
crews based at the Boeing Airplane Company, Wichita, Kansas, 
and the humen fectors staff at Edwards AFE, California, were 
tasked to evaluate the OAS. Based on conversations with 
these two groups, it was discovered that the ALCM 
retargeting procedure, also known as flex targeting , was 
Slow and cumberscme. 

mae purPpese Of flem targeting 15 to give political and 
feriyvary leaders gr@ater flexibility during both strategic 
emer tactical conflicts. In future warfare, the politicel as 
Meme as the military Situation will be in constant flux. As 
mee SitUation rapidly chenges, tergets will change as well. 
A commander’s ability to change the course of a battle in 
minutes or seconas will oe dependent upon the ability of his 
ferces tc respona to these sudden changes. This frlexibility 
depends upon repid weapon retargeting which the present OAS 
does not provide. 

Peemeossicle |S Scinution to this prebdlem was seen in 


Memputer voice technology. Woreein tnis aréa of Computer 


ie 





vcice applications research is ongoing here at the Naval 


Postgraduate Scheool by Dr. Gary Poock. If was felt that the 
ALCM flex targeting problem cculd be easily investigated 
here due to the extensive facilities of the Human factors 


Laboratory. 


B. OAS/ALCM SYSTEM IN B-52G6 


1. Background 





The B-5Z bomber haS been in the U.S. Air Force 
arsenal since 1954. Seven hundred forty-four aircraft were 
preduced through 1¥€2 beginning with model “A and finishing 
with model HH . Today only three models, the D, G, and 
"ERE remain in service. 

All three are assigned nuclear and conventional roles. 
The “D model, the oldest and least modified ot the three, 
is expected to be retired within the next two years and be 
replaced by 102 B-1B aircraft. (The USAF has designated the 
cruise-missile-capable B-i the “B-1B°. It will be equipped 
with external cruise missile pylons.) 

The G end 6H models continue on active duty and are 
the subjects of many modifications to extend their life, 
improve their survivability, and add to the missions they 
‘Meee adaiready expected to perform. This paper addresses a 
m2ssion added recently to the B-f£eéG, which is presently 
Mmeeereroing moditications which will allow it to carry the 


Air Launched Cruise Missile (ALCM). 


ie 





epee De Pave.on 

The ALCM is a small, unmanned, winged air vehicle 
Capable of sustained sudsonic flight following launch from 
the B-52G (see Figure 1). Propelled by a turbdotan engine, 
it incorporates a nuclé€ar warheed with a variable yield of 
up to <¢@ kilotons for a “hard target. kill capability, and 
Porc clammeam= Vous tl eve ns a Dre determined surface. target 
{[Ret. 1]. 

While the ALCM is onboard the B-52G, it is receiving 
EosSivVlen, heading, and altitude information from the 
aircraft’s inertial navigation system every 6@ seconds [Ref. 
Ee}. When the missile is activated tor launch, it knows 
where it is and where it is neading. 

Pewoue Launes arom tne B-S2G, ALCM wing derloyrent is 
attained within two seccnds, and engine start within three 
BecOnas. After téeunch, position, attitude and velocity 
reference aata are derived trom the ALCM’s inertial 
navigation system, mach number and pressure altitude from 
the air date system, and height above the terrain from the 
radar altimeter. Initial position/velocity data and 
eemespueric  paraneters are transferred to the missile from 
the B-S2<G carrier immediately pricr to launch. The missile 
meetnen euided inertially along preplanned flight proviles 
at programmed Mach numbers and altitudes. Velocity commands 
Pempased On Drecomputed erecund spef®ds reéquireéd to reach tire 


Target at a specified tire. 





« 
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“Time-to-target. is calculated Dyeeunc ecCOMLULer, and tbrottle 
commands are issued as required to maintain average ground 
Speeds along @ach trajectory segment. Altitudes are 
selected to be either altitude hold’, where the missile 
flies at a commanded barometric eéltitude, or terrain 
following wnere the missile flies at a specified height 
meove the terrein. 

Guidance is by a combination of an inertial navigation 
system and a terrain comparison (TERCOM) technique which 
Mit izes the radar altimeter and Onboard computer. At 
Beeaetermin@d points along its flight path, the missile uses 
Mmesmnadar altimeter to map the protile of the terrain below. 
This ‘sensed profile is ccmpared with a preloaded digital 
Tap of the surrounding area. (Mapping information will tbe 
furnished by tne Defense Mapping Agency.) Locating itself 
ween LHe Map, the missile computer determines its true 
pesition, velocity, and heading and Makes course corrections 
if necessary. At first landfall, the missile’s rap may be 
aeecam ar@a measuring in the tens of miles. wWhen it updates 
Meee pOSition, it also edjusts drift rates, calibrations, and 
Meer Buidance error contributions to make itself’ less and 
less dependent upon srapr-corrélation updates. The radar 
altimeter then goes into standby until it enters the next 
position update area. Thus, as the missile proceeds toward 
its target the maps wouid be of smaller and sraller areas, 


semee the Zuidence system fine-tunes out practically all 





error. PePcoMme is Said) to resolve objects on the ground as 


small as three meters wide and 5@ centimeters tall from an 
altitude of several thousand meters [Ref. 3]. The TERCOM 
feature allows the ALCM te fly at lew operating altitudes 
below 120 feet [Ref. 3]. Maximum operating altitude for the 
redar altimeter is 5@@@ feet above the ground. All terrain 
momrowing iS performed below this altitude. In the event of 
radar altimeter failure during terrain following, tire 
Mersosile autometicaliy transférs to the altitude hold mode 
mgr iies at a Rres@€t baroméetric- altitude stored in the 
Messitor software (Figure. i1, line #4). This value is 
selected to provide adequate terrain clearance tor the area 
being overflown. 

This high accuracy of missile position updating results 
in extreme accuracies at the target. U.S. cruise missiles 
to date have achieved accuracies on the order of 50 feet CEP 
Met. 3). This méans at least 5¢@c of the time the missile 
will land within 30 feet from the target. 

The ALCMs carried by the B-E2G are pfregrammed initially 
meanpeee oule target per missile. Amytime prior to ALC™M launcno, 
Seoew target can be entered into the missile by the 
navigator. This retargeting capadility is referred to as 
“flex targeting. Using the current integrated keyboard 
(IKB), this precedure requires approximately 1-2 minutes per 
feos le to complete. Retargéting a tull load of twenty 


missiles would require 29-46 minutes. 


Le 





Accelerated production for the USAF/Boeing ALCM at 
Mmacaueulone TratesS §“Of 480 missiles per year will start in 
Fiscal 19&4. This plan is predicated on the deployment of 
3,7E3 missiles by 1990 on a schedulé consistent with B-&2G/H 
Benvyercicns [Ref. 4]. 

5. hole of Navigator 

Responsibilities for navigation and weapens control 
Me the B=S2G are assigned to the navigator and radar 
navigator. The navigator determines aircraft position based 
On radar informretion obtained by the redér navigator. He 
momeearres actual position, course, ®tc., to preplanned 
mission course and directs the aircraft accordingly. The 
navigator is also resronsible tor ALCM status. If a change 
in ALCM targets iS directed by eppropriate authorities, he 
would be the crewmerber to make these changes. 

bicept during the flex tergeting procedure, there are no 
Meee r Inputs tne navigator must perform with regard to the 
mecm. He does have a constant missile Status of each, and 
Mmm—mone meinalcugctlions Lrior to latnch, 11% can be withheld by 
the navigator ana jettisoned if necessary. 


~~ 


pero cemarios: Tactical 6 Strategic 





The cruise missile carried onboard the 3B-52G will be 

Cafanple of pertorming non-tuclear (tactical) roles as well 
as its main role in nuclear (Strategic) scenerios. 

In the tactical scenario rapid retargeting of AICMs 


could give defending forces, especially in the European 


Ste 





Theater, an unprecedented capacity for aisrurpting soviet 
rac t Seopeyeetines, destroying support facilities, and 
meracking@ 1ront-line forces during a massive thrust across 
western Europe. Morocco wOrOlLIng cutsiage the battle area 
Semerd respond to battlefield commanders by leunching ALCMs 
meeemct targets tceo risky tor aircraft strixes, cr toc far 
amet rom drtillary positions. Since tacticel targets are 
very mcebile, a rapid and accurate ALCM retargeting 
Seravility 1s essential. os a Wa0 lec e entry system 
Signiticantly imprcecved speed and accuracy then considerable 
capability would be added to the flex targeting option. 
Utilizing a nuclear warhead iu a strategic role, cruise 
Memeo les Can Strengthen the air leg of the strategic tried 
MmeengessU,o. which includes intercontinental Dali 1b 
missiles (ICEMs), submarine launched tallistic missiles 
(SLBMs), and strategic bombers. ALCMs will give heavy 
ocmpers a standoff: capability, ana the missile’s long range 


é 


.1E&@ miles) wovid lessen the need for bomber refueling 


_— 


Byer. ¢). Planners could assign ALCM-equirred B-52Gs a 
Bena@off role of firing their missiles outside Russieén 
Meeeovace without actually penetrating, or firing and then 
penetrating to drop their gravity bombs, cr launching ALCMs 
Meeoio Russian airspace 4s they penetrate. If the rlex 
targeting procedure near cr within these high threat areas 
feemec me TCised, the Il-¢c minutes required for retargeting each 
ALCM would increase the bombers probability of detection by 


emewmy radar. 





C- SPEECH RECOGNITION & CRT DISPLAYS 


1. Qverview 





Ever since machines have been used by man, he has 

Seuedt WaYS tO heve fore control over them. With the 

development o* computers,’ man continued To exercise 

Mechanical control by means of keyboard interfaces or 

manually pvunched cards. The ability to directly communicate 

“With them ty vcice was first envisioned by science fiction 

writers of the 19&@°s and é%0’s. AS the computer became 

maioouer and able to store larger amounts of information, 

Mmesecarchrers began investigating the possibility of speeéch 

meeOeNition ty computer. 

The thecry tehind computer speech recognition (also 
known as voice data entry) was Simple enough, at least for 
meoiated words. 

The basic theory is as follows: 

Memeenir pressure variatiogs, created by speech, are ‘tirst 
Converted Into an eclectricai signal by microphone. 

2 The signal’s acoustic Yeattres are converted into 
computer machine lénguage, and stored in computer 
memory. 

Ss. Eecause a word does nct necessarily have the same length 
each time a speaker says it, the computer must time 
align its lengta. 


4. The computer then compares the features ot the aligned 





word against utterances already stored in its memory. 


PnemcomuuLer gecidesawhethner the spoken word ratches a 
Wore) Watemo ry. 

Me SSuUMmine a match is found, it then performs the action 
Or acticns (pre-prcgrammec) ccrresponding to that 


me Tera nce. 


BieweconmpureretouS  FrecOgmizes an utterance, or word, 
meermin 2 | given time freme. it also rejects any word whose 
features are too rerote trom those stored in its remory. 

meeecnmGeCoemi tion Cen De GQivided intc tWo types. The 
first is called isolated speech or ‘discrete recognition. 
Meremnunterance 1¢ fcllowed by a @.1-@.e2 inter-word peuse. 
oma aiscrete recognition system, the utterance is not 
Limited to jest a single word. for example, the Threshold 
7-608 will allow utterances up to twe seconds in length. 
The secona type is celled connected speech recognition, 
fmeriar tO normal huran sp@€ech with no pauses between 
Memos UGLErdances in connectéd speech units can thus  0oe 
reae up of many individual words greater than two seconds in 
ara tion. 

Maem ORE 2Cces rom Cemputer reccgnition of individual 
words (isoleted speech), to full sentences as in natural 
seteech (connectea speech), tae vrovlem is compounded even 
rere. In isclatec speech systems, the computer can clearly 
distinguish tetween consecutive words. In contrast, natural 


speech wave ‘fci1ms de not shcw clearly where cne werd ends 


rat 





ema the next begins, and the time alignment problem is fer 
feomenncgiiticuaey with connected speech then with isolated 
words. Nct cnly must the computer time-align individual 
freeas, it Must @lso repeat this for groups of words or whole 
ZmenrGes.  Cnce an input werd is recognized in ccnnected 
Speech systems, the computer must also decide which word is 
Mmeopelixely to follow the input word. 

ee *atue of Speecn Recognition Systems 

Cochran and=:Riley [Ret. 7] list the following 

advantages to computer voice entry: 
Mmeeeest learning time 
PeeeoUual modes cf se@nscry feedback 
©. Not restrictea to one language 
ee ACCUracy 
eee > imultanecus: Use of hands, use of eyes, data entry 
SeeeautOmatic and multirle processing 
Geenemcte entry 
Meecimpie data entry 

Pewee arane time Of a VOiCce input device 1s shorter 
than on a keytoard or typewriter, and accuracy is greater 
during the learnirg period. EBEecause spoken language is a 
Meegiras ss tart O2 lize, cata Entry using voice is easy once 
the individuel and machine are trained. The auditory and 
eee ls teecbvack immediately reinforce ccrrect input and 


meet the operetor to en error [Ref. 7: p. i9@]. 





Because the machine is trained tc each cperator’s 
voice, language end pronunciation are not critical as long 
as the operator is consistent. This mweans that regional 
accents and even fYoreign languages are acceptable for use 
with the voice input device [Ref. 7: p. i191]. 

One cf the primary advantages of the voice input device 
is that it frees the hanas and eyes for other tasks while 
lnputing data. 

Voice entry can be useful when many cifferent or complex 
characters need to be entered in Succession. The Threshold 
Mor’ Can input up to sixteen Characters at once with just 
Ome utterance. 

©. Limitations ot Computer Speech Recognition 

Although the advantages are numerous end appealing, 
disadvanteges ana limitation also exist. These include: 
1. Cost - Voice date entry devices vary in cost fror $220.92 
to.)6C30r over €§6 6987 ,2¢¢.00. As advances are made in high- 
Sieramramesratved Circults, ¢©csts will continue to 


lec reasce. 


2. Voice input can be overheard, making security a fLrobdlem 
pkec. E].. 


©. AMDIeE€nt noise interference. 
peel raining and trustration. 


Beeeeoimited vocebulary - Tne user is limited to only the 
MocaheldtyewitcieidS been Loaded into computer memory. 


Semeecnanzes in user voice characteristics can recuire 
retreining [Ref. GS]. 


Nn) 
nN) 





As in all technology, advantages and disadvantages 
exist. The final test is always the marketplace. In both 
Sovtiijan and military apdlications, voice Units are 
replecing rore and more conventional entry devices. 
Merce rarhy, @ir tradstic Control trdining, elrcraftt avionic 
systems, and military command and control centers are but a 
few or the present users. I[adustrial users are improving 
emeduction line and shipping productivity with these devices 
Meds Tuch as 40% (Ret. 14]. The Navy is utilizing a voice 
Mort at their fleet Ocean Surveillance Information Center, 
CINCPAC Fleet, Honolulu, Hawaii, to query their information 
deta base. General Uynerics recently outfitted an F-1€ 
Mmeatmeneadircratt with a pilotsoperated voice unit for 
Peet bility TEStc. 

ae GE taisSpley Techniques 

Two raijor classes Ot information diepdia / 
technologies exist today: the cathode-ray tube (CRT) end the 
Plasma panel. The CRT has existed since the 193¢°s while 
plasma despleays were developed in the late 196@’s. Both use 
fem Gt eCTent tecnhriques to display in?crmation. 

The CRT utilizes an el€ctron tear to exe Te 2 
Ehosphorous coatinog which in turn emits fhotons. tImrages can 
meen 0€ created by directing the electron bear over the 


mecespuorous-coated glass. 


nI 
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Ppeesomceciephay Is Similiar in principle to a neon 


tube. It consists of two parallel glass plates each with 
many parallel conductors. The plates are separated by a 9.1 
fMiptimete@r gap filled with neon gas. A volltege bdétween the 
mipersection of two conductors will emit a glow. Many of 
these points of light can be meade to form irages. 

The B-52G displays were required to display alphanumeric 
informetion es well as tne radar presentation. This meant a 
display technique capable ot fast updating, which the plasma 
Misplay is unatle to do f[Ret. 11]. Thus, the type of 
meso iay Selected tor use on the B-S2G was the ronochromatic 
fees Metncds ot highlighting informaticn on a monochromatic 
ae include: 

Plashing - Turning displayed infermation on and off at 
cemepeoximately 3-2 Hz. 

verse Video — Alphanumerics displayed as dark characters 
over a white background. 

meeciw1on = Position information as in double-spacineg. 

Pwembighting —- Achiéved by overwriting with the electron gun 


tTnus causing the irage to stand out. 


Peo tALENENT OF THE PROBLEM 

Pert 1 Of this thesis @€@vaivates operator performance of 
Meee - LEX targeting procedure using the present integrated 
Keyboard (IKE) input aevice end a voice input aevice. 
Orerator speed and accuracy were used to evaluate each 


system. Would voice entry improve the operator ’s 


z4 





pertormance as defined by these parameters? Each subject 
used beth input devices in loading twenty ALCM target sets 
per device. A target set. is defined as the data comprised 
fee. Characters the ALCM needs loeded in its memory to fly 
to and locate the target (Figure 11). 

Part II compares the way in which this infcrmation is 
presently displeyed (single-spaced) on the navigator’s CRT 
with twe alternative methods of presentation. These two 
alternatives are doubple=spadac ine and inverse-video. 
Characters displayed in inverse-video are shown as dark 
fissures against a2 white background. Ncrmally, characters 
are displayed as white characters cver a black background. 
Pring the inverse-video portion of Part [1, as a line is 
updated, that line is “inverse-videoed when reinserted into 
the target set. It Was hoped that this would help the 
orerator in scanning the target set in search for values to 
o€ changed out. 

meCatadoiiity Of exercising the flex targeting ortion 
Meme Voice input might significantly reduce bember exposure 
Memeenemy airspace, while giving military and civilian 
1eaders more time for retargeting decisions based on 
PecOnunmaissance, changing military and political factors, 
Etc. The method of displaying the flex targeting data on 


Mmemcoas could alsc have a similar effect. 





E. HYPOTHESES 


The follcwing hypothesis were tested: 


1. Hypothesis Regarding TIM} 





A Senere cmmiomdme,: erence in Time to 
enter complete target sets among 
the two entury modes. 

HE : Complete target sets will be entered 


faster using the voice entry mode. 


ieee nere. 1S9n0 ditterence {in Time to 
verify and correct target sets among 
the three types of display formats. 
H  : Target sets will be verified and corrected 


fastest using the iuverse-video forrat. 


<. Hypothesis Regarding ACCURACY 





Be. >; There is no difterence in the Accuracy 
of target sets among the two entry 
modes. 

H : Target sets will be entered more 


accurately using the voice entry meade. 


He > there is no dittferéence in Accuracy in 
verifying and correcting target sets 
among the three types of display formats. 

Deco are etesets will be most accurately 
Vo@mencdmands COTrected using the invéerse- 


Video fcrmat. 





Ti eee IT PTION OF THk EXPERIVENT 





A. OBJECTIVES AND CONSTRAINTS 

The purpose of this experiment was twofold. £ ipSite, 00 
determine whether computer voice technology wes better than 
meyvowoard entry for loading tne ALCM flex targeting date. 
m@eewsperronmence Cr both Entry methods, computer voice end 
the integrated keyboard (IKB), would be measured by subdject. 
many speed, eccuracy, dnd time versus eccuracy. s€cond, to 
memervine which of three rethods of data rresentation on the 
meee would te test suited vor the flex targeting procedure. 
fieweperi:ormance of Gach display presentation would be 
Pomermimes DY SULJECtT Input speed, accuracy, and time versus 
meeuracy if cheneging displeyed flex tergeting data. 

An effort was made to reconstrvct the B-5éG navigators 
Mapeeevat lod in Carrying out these objectives. Only ALCM 
Memavea CCntrecl!s and displays were incorpcrated inte the 


mocrup (see Figure 2). 


B. HQUIPMENT 


Powe varator POSitionr 





mOmeune DUurpese Of tmaiS thesis we will assume 
mar ough some communicaticns means the B-SeG crew can 
mmmecssiully receive new aALCM targeting information from 
Rrezner authcrities. mise assumed 1S that the information 


Can be reaaily decodéd and handéa to the navigator in paper 





Herm wnich will then ve manvally entered intc the 
appropriete ALCM. The keyboard the navigator would use is 
Mirue tO tne B=o2G. PeeiNveFraces @¥ith the Offensive 
Avionics Syster (OAS) which controls both navigation and 
meavpons. BoOth Navigator and radar navigator have a xeytoard 
as well as two cathode-ray tube (CRT) displays for each of 
their fpostions (see Figures 72,4). hneaethalitcy. eCltner 
OperatOr can change ALCM target informeéetion. But in most 
cases, and for the purpose of this thesis, the navigator 
MMelewoarry OUt the procedure (see Table 1). 

Maooimurarins the Operavyron of this crew position at the 
meme Sectors Laboratory, an APPLE eons (eAr 
[Mmemreomourer aya a CONRAC 7 =x S ineh CRT display were 
positioned in front of and anove the subdject’s seat 
meemypton. aA Keyboard was Ouilt to look and operate like the 
integrated xeydoard (IKB) founa currently in the B-&2G. 

The navigator’s position dimensions and equipment 
Meeetron fOr this experiment were Obtained fror the OAS- 
eeereured I-j¢ fLavigaticn Simulatcr lecated at wWright- 
memuverson Alr Sorce Base. 


eremeitesmated Keyboard (1KB, 





Pendens CO feasure target Set entry times by 
gee@ydoard, a working model of the IXB used in the B-&2G was 
built. Its dimensions ana pertinent keys were auplicated 
exactly (see Figure 5,6). The IKXB was connected via a Lear 


meee er Inc. AiM-SA terminal to an APPLE II rinicomputer. 
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Radar Navigator/Navigator Positions, B-52G 


Figure 3. 
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IKB at Radar Navigator/Navigator Positions 


Figure 4. 





The ADM-SA was needed odecause the APPLE II design did not 
allow diré@ct connection of the IKB to the APPLE II“s 
Keyboard plug directly. 
5. Apple II Computer 

An APPLE II ‘plus 64K RAM (Random Access Memcry) 
frecomputber was US€a LO control all video distlays on the 
CONRAC CRT (see #igure 7). AS Gach subject entered target 
Ser ddtd, Pic eet OmmenwOnemuase Stored in fdlin ferory to be 
Peeerec OUL ado ter Tne sessien using a MICRCLINE Micron 2&8@ 
Meee r. These trintouts were later used to check for input 
Crress . 


See vOlce Recomnition System: Teds 





A Thresnold Technology Lie T-62¢ (hereafter 
referred tO as the T6E@G) was used as the voice recognition 
input device (see Figure @). It is a commercially available 
device capable of recognizing wp to ¢€56 spoken words or 
Mmeemoes. 1 NESS EnraseS Or utterances es they are called 
Peeve Limited to @.1-2.¢@ seconds in length [Reft. 12]. 

A short pause of at least 0.1 second is requiréd between 
Es@emutlerance. Thrs the T6909 is reterred to as an isolated 
Ommemrocrete Speech VOlce recognition unit. The T6280 voice 
Goer on SyStem consists of amain terminal precessor 
Pipeeecomrdiming a Speech preprocessor and microcomptter. 
i@em=eremalning  COM~Onents, the tape cartridge unit, the 
Microphone preamplifier and display (present but not used 


during experiment) are connected separately. 





(GYI) puroqhay pazeubaquy yenqoy °s aunbry 


i) 
le 


Yup 
YSU: 


Se 


Ups 
tis 


Wil 











o4 


Working Model of Integrated Keyboard (IKB) 
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The micrephcne used was a neise cancelling SHURE SM-10. The 
TE@Q was connected to the APPLE II via an APPLE serial 
interface with RS-252 standard input/output. 

When the TEOOY receives an utterance spoken by a subject, 
the voice unit will react ino one otf two ways. The utterance 
Wumemmescompered Wirth those patterns in memory, and, if 4 
[estGmeis  ©£Ound, the coOrresponding output string will be 
tTramsmitted. The alternative is that no match will be found 
between the utterance received by the T6200 and that 
Pueerammed in its memory. if the letter occurs, an audidle 
“peep is sounded with no output string transmitted. This 
is rererred tc es @ nonreccéniticn. [t Can result from 
Pevtetmmminr Ste NOMWMGIaLiON Or the utterance, poor initial 
training of the word, background necise being interpreted by 


the T6@@ as en utterance, or lack of T6@¢ reliability. 


C. SUBJECT SELECTION AND TRAINING 


ie SCUDJ CCU spe eicT 100 





Subjects used for this experiment were students and 
feculty of the Naval Postgraduate School. Four were female 
acd sixteen male. Apeeeerca@ept tive had ne previous 
Seeeemrel1Ce with VOlcCe entry and none had used a B-EZG IKR 
Meron, tO this experiment. Seventeen were active duty JU.S. 


Meetmeary officers and three were civilians. 
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T600 Voice Recognition Equipment 
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Bemere => ne 1600NcoOuld be used as an input device in 
peece of the iKE, Yoice patterns for all input commands had 
to be recorded into the TEGQ°s memory. This is reterred to 
mee «otraining . Subjects would repeat each of the twenty 
vocedulery words (see Appendix A) ten times into the 
meeroonone. The microcomputer within the T6@@ averages the 
meoeutterences into one digital pattern of that word. aS 
Meaining was done ror each oF the twenty words. Each 
meeanllary word fiad a cOrresponding output string. Thus, if 
the TE6¢@ recognizes an utterance as zero, it would output 
a @ , and so on. The output string in the T62@ can de up 
Mmomsuzteen Characters in leneta. 

After the twenty words were trained, @€ach was repeated 
three times to confirm the TEZ2 was able to identity them. 
If a misrecognition or a nonrecognition ("beep sound) 
Mmeerrrea < out of 3 tires or greater for 4 given word, it 
wes retrained. 


o. integrated Keyboard Training 





Ali subjects had never seen an ds) betore 
perticicrating in this experiment. As Stated in the Script 
And Instructions To Subjects (see Appendix BB), the 
operation ot the IKB was shown to each subject and each was 
mueeowed Urlimited practice using a Sample computer pregram 


Mme they felt comfortable with the IXP. 





D. DEPENDENT VARIABLES 


iiemeomlowime= “Vertanles were calculated for each group 
meee target sets: 
NCE 
pelaput Accuracy Typing (IAT) So er nnn we xX 120 
NCE + NICE + NME 
NCC 
Meet Ceacc racy Iyping (CAT) = ~-~<<~-~--~~----~--- x 19¢ 
eG. ce NC 
NCE 
mesout Accuracy Voice (IAV) S eet nr nen en x 10¢ 
NCX + NICS + NME 
NCC 
Pm Outrut Accuracy Voice (CAV) ee ee ee X 182 
NG Grey Oe MC 
Legend: NCC MIMDem man COrrmect CMiaracters = 45/targzet set 
NMC Number of Missing Characters (sare as NME) 
NIC Mmumoer eer incorrect Characters 
ies Number ot Correct Entries = €& rax/target set 
NME Number ct Missing Entries (same as NMC) 
NICs Number of Incorrect gontries 


Mearaecter ~- Displayed Entries which are ultimately ioacec 


Miro tae AbGM. 


Meeey —~ An inevut which fray or may not be ultimately loaded 


mitec toe ALUM. 
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~A Correct Entries = wrerrrrerts---- » Cane Lea 
Nii + FNS 


VUipper Ome COrrect Entries = 68 fax/target set 
Number of Missing Entries (same as NMC} 
Total Number of S2ntries = 

NCE + NICH + Net TCR 


Legend: mS 
NME 
TNS 


Meeracter -— Lisplayed entries which sre ultimately loaded 
ime Om ne wLGh. 


Mercury -— An ifptt which may cr may not be ultimately lcaded 
mre tee ALCM. 


EXPERIMENTAL DESIGN AND PROCEDUR: 


ea | 


Each subject first performed Part I, the Target Loading 
Task , in which he first entered 2¢@ ALCM target sets 
beginning with either the Integrated Keyboard (IKB}) or tke 
TEC@ voice input device (see Aprvendix C tor subject order). 
Part i of this thesis was a two-way design «5 illustrated in 
Mmeeure 9. SUDJE€CL Order wath the two ccnditions was 
determined using an ABBA crdering metncd. This was repeated 
Meie the second entry mrethod with the sare target sets. 
mericrmance ct Gach subject was recorded using data sheets 


mmowi in Apvendix D. The twenty target sets used in Part J 


meercontained in Appendix EE. 


Pert if was a two-way design as illustrated in Figvre 
12. This part ctf the thesis was called the Target 
Informetion Verification Task . Here the subdject was 


recuiréd to examine each cf Gighteen target sets, ordered in 
Mmree eroups of six , which had been preloaded by the avtkor 


(see Appendix *#). 
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TARGLT LOADING 
TASK 


CONCEPTUAL DESIGN (PART I) 


Woustigia Ss 


Mumenen COmpeared @€ach tc a list in which varicus characters 
Mummmmmcoecn cheneéed. Changes varied from 2 to Y characters per 
Mme COME target sets required ne correcticns, while cthers 
required many (see Appendix G). The subject had to locate 
Sach Value in the target set to te changed, and maxe the 
required changes (see Figure 11). Only the Integrated 


MeydDoOerd (IK5B) was used as the entry device tor Fart II. 
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ome ee Ge ee eee ae Oe Oe ae ee ee ee eee ete eee eee oe Ge eee See ee eee ee eee Ge See eee cee ee ce eee Se eee Oe eee ee ee ae eee ee oe ae oe ee oe 


ie, | ! 
| | 
So | : 
S | 
om | | | | 
~ | } | | 
U co. | 
(ne | | : 
B E50 | : 
9 | 
1 18 | | , 
1. | | 
ry i. || | | 
x | | | | 
wv | | | } 
C 7 | : 
S| } | 
T 16 | : | 
fe : | 
S le | | 
1y =} | 7 
20 (| | | | 


TARGET INFORMATION VERIFICATION TASK 


CONCEPTUAL DESIGN (PART 11) 


Figure 1d 


Mrmmeor Lone three groups of six target seis required a total 
Memeoc)6CTandcmily distributed changes per gercup. e¢reuc. The 
Mmerst S€t Of six target sets was displayed in a single- 
Spaced format es is presently done in the B-5éG (Fig. 11). 
Meee Second set of six was displayed in aouble-sraced form. 


Mgeweunird set of six was displayed single-specéd, but as a 


4 


ml 


et) 


mene was modified, that line then b€care dis 7 e0 ea 


Aé 


Mmrerse-videc fcrm. Uicwecters displayed in trverse-videc 
MmemsnOwWwn eS derx figures against a white background. Koha 


meueilS of the experimental precedure, see Appendix B. 
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ALCM LOCATICN AND ILTENTIFICATICN CODES 
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RETARGETING PROCEDURE 


ACTION PESPCNSE 
Desired ALCM is selected eevee = PON. Clo nev 
by navigator. Peeuor sdasplays target 
information as shown 
in serene 11 . 
meiner l 1S Selected to dE Coen we 0S Te peated at 
enanged. DOL UOMr  OleCa! p,eadcy 10 
re changed as shcwn in 
Pe Uete alder. 
iMmewecadtad 15 Entered using Sew 1 ne 4) aupedrs at 
meyeodrd sor lire #1. DormrcMMOraGr as it is 
Typed, oOVerwriting the 
Srie inal line 4, 
New line #1 is cempleted; ee ome ne rls replaced 
ENTER cutton is pressed. by new line 41. 


C 


ieeoe Fe 16 selected tc be Pinte tevltcere pea Led 2} 
changed. DOr von Of CR1 reed; 10 
Dee-C ma Need. 


meas Drecedure iS Continued until all “ive Lines have beer 
Beene ed Out. After the last line ( #5 }) nas been entered, 


maeeabct! is réady tor launch against its new target. 





imeem s es tedatceth, Subjects called ur Gach target set as 
memcone in the B-o2zG using a Llétter—-number ccmbinaticn. The 
meee seSlOred Under the Lert alrcraftt rylon are referred 
Ge as Al throveh AS. The Gight stcred in the bemb bay are 
mererred tO as Bl throveh BS. The six ALOMs stored under the 
Meru aircrait pylon are referred tc as Cl through Cé. 
mesure iz SnowS Lictorally the idé€ntification and position 


Geeeach ALCM. 
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A. INTRODUCTION 

Restlts for Fart I (Target Loading Task) and Part II 
(Target Information Verification Tesk) will be discussed 
Meearacely. For €ach, the resvilts are suodivided into Time, 
mmemee and OCutLuUt Accuracies, end Time versus Accuracy. Raw 


Sava analyzed in this secticn is ccntained in Appendixes 4 


eed «6. 


Pee a2cULTS OR PART I 
ie Lime 

Pee item smo mpOol Smewine the tire required for 
meer SUDjJECt te lead all twenty target sets. Keyboard entry 
_o~eeorlelr tiaea tile VOICE entry technique for tifteen of the 
Meemeteen subjects. LOmpceeMCe rence @fes 2oted in the tine te 
Mepee te USing Citmer technique for one subject, while voice 
MPMeNd ceeds her 260 Leree subjects. [he five subjects at 
Memon rifnt Of tne Eraph head fErevious exverience With 
merce entry. Picioyisn ome weitere 15 jndicates no marked 
Memeeecice @amMO0ne Those wild prior voice entry experience. 
mean and standard deviation tor the voice data are <7.5€ and 


Meemeeresvectively. Mean ana standerd d@vietion for tre IKB 


Mee are 2<4.C4 and S.15 respectively. 
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Time (nearest min.) for Voice 


Time —---k = Time (nearest min.) for IkB 
(Minutes) B = Voice Tire = IKE Tire 
* = Previous experience with Voice Entrv 
K 
4¢ 
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Pees ON VARIANCE: SUEJECT FY TIME - PART ! 


SOURCE SS) df MS P 
Hegween Subjects 412.42 1S ashe Agll Le ns 
Retween Metheds gor 24 I! 85.24 Soe Kage 
eecor Clie ieee i 16.47 
Petal elle ore 39 


iot-m merce ratea “Neyooerd Input method proved te be 
Memeo ticaktly quicker than tne vcice method as shewn ia 
Mmeple Il (P <.g&). 


Zee input Accuracy 





PeImen ce SON SmoNoyecCt by Percent input Accuracy. 
emi Percent inptt accuracy was higher for 74 percent of 
MemmouoyeEcts. VYolce wes better for ¢5 percent. This wes 
Memmrrrred using an Analysis ot Veriance shown in Table I[Tf. 
Meesinout accurecy wes significantly better (P < .@1) than 
vcice entry. PeGorGer tO Feet Tequiremrents regarding the 
@rstriodutions cf the date, an are-sine transfermaticn was 


femeed LO THE Lercent input accuracies frior to perfcrmineg 


meee Analysis c* Variance.}; 





TABLE IIT 


Memeo Ce VARITANCH: SUBJECT BY INPUT ACCURACY - PART I] 


SOURCE Ss Git MS ¥ ie 
Between Subjects AS eC Sls, eo 0.0505 ee il 
begween Methods Ceeeoe 1 Qiccce 10.99 CaO 
zrrer Z.4583 is Wieee | 
moOta 1 eae ed OY 


seeeoutput Accuracy 
ieee Seco mOwsS me OUD IECT “VETSUuS FErcent Outout 
mecuracy. hicepotc Ol weyarlance in Table iV confirmed nao 
Saenizticant difference tetween voice and IKE percent cutput 
accuracies existed. (In Oordér to meet requirements 
Mereraing the cdistributicnrs ef the data, ar arc-sine 
MmeamatOrmation was applied to the percent output accuracies 


mercer tc perfcrming the Analysis ct Variance.) 
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TABLE IV 


ANALYSIS OF VARIANCE: SUBJECT BY OUTPUT ACCURACY - PART I 


SOURCE SS dz ae 


¥ ie 
Between Subjects Q.7Y4E ILS) 0.24165 <1 ns 
Between Methods 0.2214 1 Z.0214 out ns 
Error Ik Cope}, ig 9.9558 
Total lpr a haere 2g 


4. Time Versus Accurécy 
Figures 16 through 19 show Input and Ouro > 
Accuracies plotted against Time tcr both voice and IKB entry 
rethods. Figure 16 shows no apparent tradeoff between 
Mmercent input Accuracy and Time for voice entry. However, 
Figure 17 through 19 show pLronounced increases in accuracies 


memeoime to corplete the task increased. 
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Meeks ouGls FOR PART {I 
1. Time 
Figure c@ shows everage times for each of the three 
mecoplay formats tested. Mnalysis of Variance between the 
three displey formats shown in Table V indicated no 
miemiricant differences. However, a significant difference 


between subjects (P <.€1)}) was detected. 


TABLE V 


ANALYSIS OF VARIANCE: TIME 


apo toe bnerOmmats = PART 11 


SOURCE 38 df ate ¥ e 
Between Subjects EG epe i9 1.977 4.48 od 
merween Methods PLS ‘a Doe t 1.49 ns 
mercer ike Byee SE 0.44 
aro La ) Sots oo 
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Peep teen eC Cura c Y 

Wah viiels re, Se shows Tom eeel ene cunacy plotted 
against each Display format. No significant difference was 
mound DELWEED display tormrats as shown in the Analysis of 
Variance in Table Vl. (In order tc reet requiremerts 
Geeoreing the distributions of tne data, an areas ine 
tTranstcrmation was applied tc the percent input accuracies 


prior to performing the Analysis of Varience.) 


TABLE VI 


ANALYSIS OF VARIANCH: SUBJECT BY 


Wiel eaGCuNn Gt = PART [1 


SOURCE Ss det MS 4 P 
merween Subdjects Q.940E 19 @.@495 SLES ns 
Petween Methods Cae Zoos ae A Sis <1 ns 
eerer 1.71¢5 Se @.€499 
eta ] LC oY 
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So VOutpUL ACCUraCy 
Figure ize snows Mean Output Mecinecy “colo t ted 
against Each Display Format. No significant differences 
were computed as skcwn in the Analysis of Variance in Tatle 
Merl. (In order to meet requirements regarding the 
Mer eroutions of the data, an arc-sine transformation was 
meolied tO the percent output accurecies prior to performing 


the Analysis of Variance.) 
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ANALYSIS CF VARIANCE: SUEJ&CT BY 


CUPEUPMACCURACY — Pant: II 


SOURCE Sve df MS F P 
Between Subjects FES 3h, 1y G.13E ae ns 
Between Methods Coe 2° 2 2.48 <1 ns 
ecror 3.189 o& 0.884 
Total Seve Sie, 
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mice colomet Vallance L£Or SUbjJECt tires and input and 
eutrput accuracies sSnewed Demect 1 Ca0t CGilT ferences. 
MeeeveT, plots ot Percent Input Accuracy versus Tire tor the 
Seemees dicplay itcrmats indicate the dcutle-svace format 


Beewrvec in the best percent input accuracy over time. 
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Figures 23 atd 25 show data scattered over wider ranges a5 
Mmempere€d with tigure <4 which shows gooa clustering above 


mae oo percent level. 
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An examination of the Percent Cutput Accuracy by Tire 
plots showed no significant grouping otf the data in all 


three displey formats. 





TV. CONCLUSICNS AND RECOMMENDATICNS 


ee eee 


A. GENFRAL 

The purpose of this thesis has been twofold. Part 
investigated the use cf a discrete voice entry system in 
loading Air Launch Cruise Missile (ALCM) target information. 
A direct compariscn was made between voice entry and the 
integrated keyboard (IKB) presently operationel. Time to 
lcad, input and output accuracies, and time versus accuracy 
fmormrerentsS were recorded for each of the twenty subjects. 

Part II iavestigated three means of displaying the AILC™ 
terget information on a CRT. Information wes updated using 
single-spaced, doutle-spaced, and inverse-video formats. 
Mary the IKE was used in Part [J]. Time to update, input an 
Output accuracies and time versus accuracy measurements were 


meeorded for ach subject. 


eee CONCLUSIONS 
ime Part. [ 

Tt had been hypotnesized that target sets would be 
emvered taster using the voice Entry methed. However, the 
IKE was significantly faster (P < .@&) than the voice 
rethod. It was suspected that this may have teen due in 
part to first time use ot the voice entry device by fifteen 
Gu t ‘Chea twenty Sete ols. vue tc experimental time 


memmeraints, Sudjects had at most 1£&-39 minutes to practice 


ev 





mureevoice @ntry prior to running the experiment. However, 


analysis of Figure 13 shows that only two out of the five 
pee os with prror voice Entry experience accomplished data 
mmuryeduicker uSing voice. Thus, prior exposure to voice 
did not improve subject entry times. It should be noted 
that despite slower input speeds using voice entry, 68 
Mmemecemt Of the subjects preferred volce entry use over the 
eB. 

ieesnoOuld be NOtTEd that @ discrete volce unit was used 
“cr this experiment. However, a ccnnected speech’ 
recognition unit would have been more appropriate for this 
task, and oprcebably would have shown faster input times. A 
Meemiec ted Speech system is able to receive multiple inputs 
at once, ie., 1-4-3-2Z-5', with no pause between utterances. 
mieemmore naturdi speech would te more comparable with 
memooard Entry than the discrete voice entry. Unitortunately 
such a system was not available. 

It was hypothesized that target sets would be entered 
rore accurately using the voice entry method. The IKB wes 
meenisicantly higher in inpet accuracy (P < .31) than voice 
entry. Thus fewer corrections were made to data on the 
memeeen CETOre committing it tc the weapon using the [KBE 
eaiury technique rather then voice entry. 

MUGuMer facler which etfected input accuracy was the 
Memone Of the voice unit by the subject. Eech utterance 


femeeeecrca ted TOree Limes in order to detect poor training. 


Es 





If the voice unit had problems recognizing an vtterance 
mepner diver tGraintame Or during practice, that word was 
retrained. In many instances it was not ncticed until well 
mimomnne EXperiment that certain words had either teen 
Peeriy trained or were being prenounced differently by the 
Svojyect. Changes in pronunciation mey also be attributed to 
the stress impcsea by the experirent which was not present 
during the vocebulary training session. Regardless of the 
explaination, subjects made less errors using the IKB method 
of entry. 

femcolenitiecany adititerence in The output aiecuracy existed 
between input devices. This indicates the data was checked 
equally well by each subject prior to entering the data and 
moving On tc the next weapon, irrespective cf entry 
meennique. 

NWO apparent tradeorf was noted between Fercent [nput 
Accuracy and Time for voice entry (seé€ Figure 16). However, 
pronounced increases 1n accuracy were noted for Percent 
Input Accuracy by Time using the IKB (see Figures 17). 

Possible fatigue was suspected during the voice entry 
portion in Port ee Supsequent analysis of all 
Meer ecoecnitions and nonrecognitions over the twenty target 
sets shewed an increase in average errors as the the 
experiment progresséa, but tas increese was not 


Statistically signiticant (see Table XIII). 


os, 





Pee Parte La 


~~ 


Tt was hypothesized that target sets would be 
verified and corrected quickest and most accurately using 
the inverse-video format. Analysis showed no significant 
mpecerences in time, input or output accuracies, and time 
versus accurecy between the three display formats tested. 
This was rather suprising. It was originally telieved the 
Peenele—-Space fOrmat would require much mcre time and induce 
rore errors than the other two methods. There were 
mrtferences in subject preference among the three display 
formats. The double-space format was freterred by 65 
percent, es compared with <@ percent for the inverse-video 


and 15 percent for the single-srace. 


C. RECOMMENTATIONS 

Eased ON the data in Part — it can ve concluded that 
while voice entry is definitely preterred, the IKE is 
Superior in entry speed and input accuracy. Due to limited 
mescurces, this thesis had cnly a discrete vcice entry 
device tO measure against an IKB. It is the author’s 
cpinion that a coré€cted voice unit would have shown much 
greater input speeds than the aiscrete unit used here. 
Mans, voice applications in areas requiring quick lcdéding of 
Single characters Should look toward the continuous’ voice 
meruemS fOr improverent over conventional Feytoard entry 


devices. 


7 





While there was no significant difference among the 
moaree display ftOrmets, it is suggested that the aoubdle-space 
display format be used. Use of this format would incur no 
eeerulonal cost and would satisty user preference. 

It might be desireble tc cempare the effectiveness of 
the Loree display forrats iimuier Suo,ect ~~ voOSeda and 
Experimenter posed conditions. Thus, the effect of stress 
@euid be simvlated. 

A briet examination of nonrecognitions in Table XI¥ 
mareated that the terms 9, <2, 4, end 5 were more often not 
Mecoenized then the otner numerical terns. A MSs was 
merrOrmed on the frequencies observed for the 9 through Y 
eeuries and a Nice value of 137.52 was ottained (1@ df, P < 
m1). The © throveh Y valves hed been drawn froma table 
Of random numbers, thus e@ach had an equal probability of 
being misreccgnized. The data suggests that the entries 
feet, and S are over-represented while the entries 9 and 


S are under-represented. 





APPENDIX A 


T6@@ VOCABULARY LIST 


TE6Z@ TEC@E 
Lecation Output 
Register 

GUO zero 
OO1 one 

O€z2 two 

OOS three 
004 POUL 
ZEZ5 five 
OY6 6 1x 

OO seven 
G28 eight 
GOY nine 
019 contno | 
Q1il Teo: 1 ¥ 
O1z enter 
2135 yes 

G14 no 

aie alfa 
G16 brevo 
Gi” Ccherlie 
G1& Left 
019 ae 


We 





APPENDIX B 
SCRIPT & INSTRUCTIONS TO SUBJECTS 


1) The CRT and the integrated keyboerd (IKB) you see 
here are similiar to those found at the navigator’s position 
in the B-E2G. I will ve measuring your ability to load 
target information in actual ALCM formats using keyboard and 
voice entry. The experirent is divided into two perts. 
Part I will require you to load ¢<@ ALCM target sets using 
Boun the IKXB and voice entry. Part II will require you _ to 
call up each of six ALCM target sets I Lave preloaded, and 
make corrections to ther if necessary with new values you 
Mell have in front ot you. This will be dore three times, 
Sach with a different CRT display format. 

2) Show photos (Figures 3,4) of navigator’s position in 
B-52¢. Show actual IKE (Figure 5) and the working model 
used in experiment ‘Figure 6). 

Bye if the veice unit is used first, train the 20 
Mecadulary wOrds at this time. Explain the theory ot voice 
ertiry. <Aiter words have been trained, have subject repeat 
aero tonree times, confirming the voice unit can understand 
at least two out of the three utterances. For, each wword 
Zone the above test, have subject retrain that word. 

4) Show subject typical ALCM target sets. Run practice 
Memputer prograr. SOc jeGue Low to call up, enter, 


emerr VY, EtLC., target sets. 





VOumebaverdacemucm Lime as you feel is necessary t0 


Eractice. MoigumyOl feel Comfortable with the procedure, we 
Will proceed with Part I cf the experiment. 

(Answer any questions and assist if necessary.) 

5) You will be given 2@ ALCM target sets printed on 3 
sheets of paper. When I say, reaay, begin , enter the sets 
as fast as pessibdle while meéintaining accurate inputs. iG 
you make a mistake, be sure to correct it. 

Any questions? Ready, oegin. 

6) Repeat above paragraphs (4) and (8) using other 
mmery MeLnNOd, atter sudject completes a ten minute break. 

7) We will now begin Part II of the exyperinent. I have 
Meeroaded 12 ALCM target sets in three groups of six. fYou 
Peeeeebe £iven Each set of these six target sets separately 
On a sheet cf paper with scrme ct the values changed. You 
will be required to display @ach target set on the CRT, 
menmpere it to the correspcending target set cn the paper, and 
[eee tne necessary corrections to the target set on the CRT. 
some target sets will require no corrections, while others 
fmeeerequire many. Each group ot six target sets will be 
feesplayed on the CRT in cne of three ways: single- speced, 
memurple-stracéd, Or inverse-video. 

Show subject example of ALCM target sets with changed 
frees Of a Sheet of paper. Run Lractice computer program. 
Alicw subject to practice until he is comtortable with the 


procedure. 
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(oUmoavesassm@ucn tine as you feei is necessary to 
practice. nen you 2eel cemrortable with the precedure, we 
MemmeIrOCceed With rary [l of the experiment. 

(Answer any questions and assist it necessary.) 

&) Repeat item #7 if subject fails to understand Part 
II of the experiment. 

If you make a mistake, be sure to correct it. 

Any questions? Ready, vegin. 

9) Repeat above paregrepks (7) and (&) using other 


emery method. 





APPENDIX ¢ 


EXPERIMENTAL ORDER FOR SUBJECTS 


Memb lo 1,5,27'se,ll,15,15,17,19 perfcrmed Part II of the 


eeeperiment in the following order: 
fae seu TOC hommes hk = Voice, keybtcard 


memmols 2£,4,6,€,10,12,14,16,le,20 pertforrec Part I! of the 


eeeeriment in the following order: 
iapee tebeadiie  laske— Keypoarda,Voice 


Meee 1,2,0,0,6,7 ,9,11,15 performed Part II of the 
Maeerifent in the tollowing order: 
Target Information Verification Task - 
Sineile—-o paced, Wenuole—Ssnaced, Inverse-Video 
miegeGls 15,17,19,4,6 performed Part II] of the experiment in 
the following oraer: 
Target Information Verification Task - 
DOubve=Speced, Ilnverse-Video, Single-Spaced 
Seo eols 19,12,14,16,1&,2¢ rerformed Part sea Out the 
Seeceriment in the follewing order: 
Target Information Veritication Task - 
Inverse~Videc, Single-Spaced, Double-Spaced 
abe < wen ee wGr ty SUD JECTS performed first 
Part 1, Target Loading Task . This 


was followed by Part Il, “Target 
GEomEOEMat bon Verificetion Task . 


Ee 





APPENDIX D 


SUBJECT DATA SHEETS 
TARGET LOADING TASK VATA SHEET 
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TARGET LOADING TASK DATA SHEET 
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TARGET INFORMATION VERIFICATION DATA SHEET 


CIRCLE ONE: SINGLE-SPACED DOUBLE-SPACHED INVERSE-VIDEO 


ACCURACY 


ame oe =e Ge = ae ow 


en en & Gee oe] a= 


NME = NMC 


14) 





TARGET INFORMATION VERIFICATION DATA SHEET 


— a OE oe ee ee eee is ee oe —=o ee eee eee ee ee ee oe ee ee ee es ee ee ee i oe ee ee eee ee ee ee ee ee ee a= a = oo — oe woe oe ee —_— a => — 


—_— a ae ee cee ee eee ee OO ge eee ee ees ee ee eee ee ee ee ee ee ee eee eee ee oe ee ee ee eee eee eee ee ee eee ee eee ee es ee 
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ite 
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(eon te. 


TARGET INFORMATION VERIFICATION DATA SEELT 
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APPENDIX f& 


TARGET LOADING TASK INPUT DATA LIST 


N51864725 
8467214055 
1965458875 
97051E87459 
E1498 


WEAPON Al 


(NPA NR 


N1i9@574285 
GUreCe Goe. 
1£189547255 
Beer eco. 1 
4091l<e 


WEAPON Ae 


nee CARD 


NE6872130 
Bi2av9o7264 
Pee (eote’ 
07 S$6275E564 
2eloe 


WEAPON AS 


(nib (CN NS RR 


WEAPON A4 N&Z659574 
VsIeocze 186 
5671946572 
cESEES1IOEES 


1432 


(nem CA NG ke 


WEAPON AS N1@725497 
BY 2e002o47 
1SS0W2457 2 
BOLeIat ool. 
cE856 


ine CAM 


WEAPON AE N16500364 
BEB CS75724 
0754712041 
16406457555 
69858 


nib GN ere 





WEAPON 


WEAPON 


WEAPON 


WEAPON 


WEAPON 


WEAPON 


WEAPON 


Bl 


Be 


BS 


R4 


Be 


Isle, 


BY 


ni Ce Mm re Om CA ND Re CWP CA DS & ne (AN (TP CA DIF np NF 


MP CAN 


N17896554 
E194653855 
9481937906 
64868551441 
S65s7 


N52849654 
E167452749 
LiGWelaeis reislaig¢ 
Geosoe (2051 
20756 


N13474628 
E285658361 
6120S4E24E 
PeSeges (oc 
6582e 


N@14728ES 
¥184644275 
I Veeco (2 
G7 4528659575 
reowe 


NE&6612086 
4681902642 
1943028474 
G28a791E491 
Ogee l 


N10<258475 
4827561856 
7495298156 
06104829154 
E8264 


NOE746S8e 
Were Losoo6 
9482127462 
128£5485217 
62741 


E4 





WEAPON 


WEAPON 


WEAPON 


WEAPON 


WEAPON 


WEAFON 


WEAPON 


Be 


C1 


C3 


C4 


CE 


(Nf CA MD COP IN eR CNP CA ROR (NP CAN Fe CnoP ON NO kr ini CARI 


nib CAN 


N582861¢& 
E1GSE56ECE 
Pee olgs ¢ 
18401746218 
918735 


N10554826 
£192462651 
629501647z 
CS741845<26 
iL hee 


N17494625 
Hewes (aU 
4612574628 
MWeEbvoel sol 
Mees 


N56194@15 
E450c8 OE46e 
Gigs2os726 
Plate ooecos 
7O08E 


NO17E5476 
£107562743 
165&2E4654 
Sigcec(1ve4 
612828 


N1674197¢ 
mUOCSSS? 254 
9629621652 
sivecsezol4 
E€<614 


N&69527535 
¥1065E567<c 
1E5ES05647 
Olroecoo7 81 
aiken’) a 





APPENDIX F 


TARGET INFORMATION VERIFICATICN TASK 
PRELOADED TARGET SETS 


Summary of #characters/line to bé@ changed out: 


seven hundrea seventy-four total characters are possible 
tO be changed out in all 18 target sets. A value of 1@% was 
@aesen to be chans@€d out by each Subject. Ten percent, on 
774 = TE, the number chosen to be changed out. 

All eighteen terget sets were divided intc three groups 
of six target sets/display method. Each of the three grours 
had c<€ characters tc be changed out. Characters to be 


Seansed Out were selected randomly. 


#Characters/line to 
be changed out: 


N@27463&e 
EO781905856 
S4621074E2 
12&25485217 
62741 


WEAPON Al 


(rT PON NDR 
YQ QR QO 


N16554826 
BE1¥2462651 
6295916472 
597418452e2E 
Lav (ee, 


WEAPON Ad 


ON of GN FR 
Rr QR PY 


ee 
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WEAPON 


WEAPON 


WEAPON 


wEAPON 


WEAPON 


AS 


ne 


Bi 


Bie 


ON BP CA NI Fe (NP CA NSH On (ANG OP (AN Re (MH CAM 


Ob WN FR 


NE ZE59S74 
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SE71946572 
cESEEEIBOEES 
1o4F2 


NZ1864725 
E467 214285 
1965482873 
979516874359 
6149E 


N13474638 
ECBSE6GESE1 
6120546246 
Cleeve s( Se 
65622 


N@147 28635 
8184644278 
UG ciaielever ates 
07452865575 
ToS oc 


N17494625 
Heedoco1 Mr 
Senor etce 
PST (Sel 561 
eG 
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812¢957264 
Pegarco4be2 
G7 SCe CCoCS 
26158 
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#Characters/line to 
be changed out: 
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WEAPON 


WXAPON 


WEAPCN 


BS 


B4 


EG 
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B& 


(im GAR mMmMioeP tA ADR (Neh Cc. NF (YP CN DDR (Nh (A DIR 


CHa (A ADR 


N19258475 
¥Q0273561E56 
745298136 
08164829154 
68264 


NGG clea (aay 
E19ES8567 2 
1E€55905647 
Ura oe Zoorel 
01558 


N174946c5 
BG629575107 
4E10S74EZ8a 
ver coSlcel 
Hel oleys) 


N19&ES7455 
POW SeSC5co 
1214934755 
BeereoolUy 1 
4091c 


N17€96354 
H19465S5ECS 
S4519579GE 
6486651041 
S6597 


Neeweo lve 
B19365682E 
Poo cools ( 
1649174E21E 
91273 


#Characters/line to 
be changed out: 
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CS 


C4 


OOP NM Fe OP OMe CHP ON Nr 


(NP CA Mr 


Zier 2ees 
¥164644276& 
Peeeoooc (5 
OV 452660575 
Pesce 


N16725497 
bGecoocos ¢ 
PeeOeeto 72 
WONBIS1 SIO 
c£E€8S6 


N52849654 
2127452749 
1275382€57 
62956272651 
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Nisei S4915 
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Sig 2coeaot 
Weeee 
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#Characters/line to 
be Changed out: 


QQ QFR YQ NO OF NY YQ QB 
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WEAPON 


WEAPON 


WEAPON 


APPENDIX G 


REVISED DARGET SETS TO BS UPDATED IN PART II 


Al 


Ac 


Ad 


A4 


AS 


A€ 


TARGET INFORMATION VERIFICATICN TASK 


(Po CAND (Nee WN MNr Nb CNN FR Ooh CA DD (TP ANP 


Cn (xn Mr 


SINGLE-SPACED DISPLAY FORMAT 


NO7 442594 
were LeSec 6 
59¢82127462 
1G8e5485217 
ear 


N1¢254826 
h1Oc4€c671 
Seca toerec 
9741845226 
Lu aiee 


NEZESOIS74 
Jf) SleisvelSyeeis 
Si Cogteor 1 
<G@366610564 
18452 


Neleear2o 
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196545¢475 
9Y<UW16E7435yY 
614Y9E 


N1342468 
PiSseoeo cl 
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oc&cg 


N@1472663 
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WEAPON 


WEAPON 


4 XAPON 


WEAPON 


WEAPON 
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Jail 


BS 


£4 
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B6 


CNP CA MI OP & NF Cn ® CA DOR (NP CAN FR On GNF 


(Nm CAD 


DOUBLE-SPACED DISPLAY FORMAT 


N176946<5 
I[Svals)D) (drone aie 
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Pet 6Se15ce1 
ISD) So 


NSE6E7 LESS 
E@229572E4 
eI O(2o262 
Gvecer sed 14 
ZO 1Se 


N 16258575 
#17 5540855 
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96164829154 
CEc<E4 


Peele o0 
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Giveecoous | 
21548 
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weEZzOs col? 
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ijee (SE1SE] 
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aot ec 
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WEAPON 


W2APON 


WEAPON 


WEAPON 


WEAPCN 
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Ee 


Cl 


(rTPA NF (NP CANS (IP (NN FP (MPA Me (nie CN NR 


CrP CANS bk 


INVERSE-VIDEOQ DISPLAY FORMAT 


N’7 896554 
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oat o7 176 
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Saco / 
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E19E6566z8 
VG CoCo 
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Ser 
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GIGS a ot aie oe 
iLeletey 
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UC st., 
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APPENDIX & 


EXPERIMENTAL VATA 


TABLE VIII 


SUBJECT ERRORS — TARGET LOADING TASK 
Crk ee ) 


—_ = Ge a= eee Ge ae eee ee co ae ce = oo Oe Ge ee OD ee Cet ae Gee Ge Se Se ee OD ee Ce Oe ee eee ee Oe ce eee ee OO ee ee Ss Ss Se 6 Se SE Se Se Se Cee eee ee Oe ee ee ee ee a en a 


meee NIC ! NEC | NICE ! NMC } NCE !| TNE ({% CORRECT ENTR. 


me, © ! gy | Cee Jetoewe) Vso1. | 497.6 : 
2 See: See Cree eo | eerie | 1448 | %90.9 
me, 2 |} 6 f Gee) we) | bee 1578 | 498.2 
PeeecCen meee (St chhhae 6h seg) | 61375 | CoA 9 
cee; 6 6©6flCUC |] CUCU] See | oleees, 1444 | 493.6 
coo) | el Oh} S!!hCU] USS | 61876 | %97 9 
moo | 6 ! Geese 7) Gleeay! 1371 33 %YEE : 
eect || 64S Uh AG!) CU! es | 6uises SU! %93.2 
eee ry tll l|lhlpehlU|tl l6as4ee |} 1299 | %94.5 
EE NS es) ee aes 2) res So oe cl om %47 4 3 
mr, 6USlC]lCUCidGCtC*?! ies roe | eteso |; eer. } ~99.8 
mama fl UU s!!hCU}lC CUCU] UOC} ce9e | 1874 | Lee. 7 
fee, cc | 4 | Amine: | TSGe | 1eeEee | 499.1 : 
me! 2 ; fs | See raoe -, eicee |) 1694 | %S7.2 
Se 2 (0 %93.2 | 
Merce oj ly”6= | 6 8mmSl!lC Sl AU! Cf Cee? | «(C1se? | %Y91.2 
ae C.F ieee Oo esos, eee | ~9E.E 
cS i es ewe jess | sey | “OE LY | 
ST yy ee es Us GPG EG Sh ~94.8 
woe i 2 |! g§ |! Sue fon) Mie. i eee | %G7.8 


— oS oe ee es es ee ee ee es es es ee ce ce ee es es ee ees eee eee es es ee es es ee ee ee ee i ie 
———— oe ee ee ee ee ee ee ee ee ec ce ee ce ee ee ee ee es ee es ee ei 


Meoanob: NIC: Number Incorrect Characters 
N2@G. Nember #Antries Corrected 
ViCtew Umer eGo incorrect Entries = NIC + Nac 
NMC: Number Missing Characters (sere as NME) 
NWO: Nurbper Correct Entries 
ete Otvd. Number Srrers 


go 





TABLE IX 


SUBJECT ERRORS: TARGET LOADING TASK 


(VOICE) 


om w= os oe Oe Oe oe CSF Oe Ow ow 6 Oe SE SS ow C2 © SP C= SD © C= 6 C= = SO Se SS Se OS ee SO SS SS 6 S|) © Se © © 6 © 6 6 6 © © 6 6 = 6 Se Se 


= gee Gem Ger Ge oe gees oe @ ww CP Ow & OS 2 = Owe ew PS Se C2 ES SS SES SS SS SSP) SSeS 2 6 Se Se ee 6 So Se Se ome OP ep ee @ee@n = = = OE aE ee ae Ge a= a= 


ee eso cor. |  G | JYzZz1 |} 1268 |} ~99.2 
eee err 66 '| 7 | 136¢¢0 |} 1412 | %91.6 
teers) 40 1 g ' yeee ! lars | 491. | 
eee meee! aoe 1 oe 6! asd?) 6! «1e98 (|! oeac 
ee eco Gee | ~S° j-wee7 | 1442 } ZEY.L 
mee Gece | 47 tl 2? lo aSis |! 1é92 | %Y93.9 
eee eo || eelhCUl GC] 1g | (1420 |} L265 
eee, S22 4 45 | 1349 ! J4a¢e | 492. ; 
72 SR: Ss Se ee 2 a SD %89.3 
eee cces!hCU} CUCU ELCUTCUCQCU CO BO9D } «C1440 «|! 490.9 
opie , ci | 6) | ¢ ! je79 | 1418 |} “LOC .2 
aoc, )6,f ckGlhCU SC YCUlC tt‘ Q|séSCQQTE UC CU1S99 ! 494.1 
Smee, GlUC]:C(<‘<‘i‘(<‘<‘ ‘;;:#C‘CwKYDSCOCS}] HEC CU CU1819 | 1878 | aoe. . 
weet | lCUGlUC<“ T;‘“‘i hl U:h6h! Classe ! 1292 | 496.1 | 
feo UflC<(“ ;:C‘CSSUD}CUBGULUCUh! CCU] O1E71 6! 144g | yay al 
mom; ¢ ;, lr | 108 ;,; 2@ |; 1262 | 1444 } YEE .7 ! 
Rumeeco ,; cr ;, 128 | € | 1242 | 14356 ! Lees 
Pee |) te tl lh 664m eh 6hlCU! is | 6fago UC 493.5 
meet 6h} Ul! CULL! See | 1891 ! 495.2 
omer’ | 22 {| 1129 | #g j 12&@ | 1449 ! Rea 


— Se ee cee ee ees ee ee eee ee ee ee ee ee ee ee ee ee ee eee eo ee ee ee ee ee eo ee ee ee ee ee ee ee ee ee ee ee ee ee eo ee i ae 
Nc ce ed ee) 


LEGEND: NIC: Number Incorrect Characters 
NHC: Number Entries Corrected 
NiCh: Neimper of Incorrect Entries = NIC + NEC 
NMC: Number Missing Characters (same as NME) 
NCE: Nmmber Correct Entries 
TNE: Total Number Errors 
Peer eevlous experience with Voice Input 


(The voice portion of NICE also includes the number 
of voice unit nenrecognitions and missrecognitions) 


v4 





TABLE X 
SUBJECT ERRORS: TARGET INFORMATION VERIFICATION TASK 


(SINGLF-SPACED FORMAT) 


o_o ome oe oe ow we we we ee owe = ew oo oe oe ce CO ee eee So Oe ee ee ee Oe ee oe ee ee oe 6 oe oe ee oe 6 ee ee © ew oe ee oe es © ee we © eo Se ee 2 ee ee ee ee 


meee, NIC | NEC !} NICE | NwC | NCE | TNE {% CORRECT ENTR. 
i, ct ae; ee SS See! eS ys Le PANS hal ! 
2 rn eo jee: | Ce %122.2 | 
ce; «2 | @ | BT le ae AS) A 
2, |; a Cee | eae 1 as 'yiseg 4192.0 
= i: owe Pecos erie. 1 1560 =| 492 .2 
ees ie eee ecudelo, ' ise 3 %98.5 
? a ne ee Gio oes | lee! Cf %Y9E.5 
cme ot , 2 ie iice jee | ise. I 92.3 
2 La a Guo ee se %10¢.2 | 
m , @ ; 1 | ee Cees), ee | ye) Hd 
mom; © |; &€ | Se eeler jesse | Tas, | %84.9 
im |; @ | @ | CEP Cee ee eee! %102.2 
m ;, ¢€ | 1 | oP eee 7) sie yO 
mee, 4 +} 4 } Sno |) 6Cfweiee | dee 434.7 
im i go |}; ¢€ } Scie eee, te ioe” Ct %E4.6 
fm ; 2 |; 2 | eles ees = sige ! %9z.8 
mee, 2 i! e2e | eee eee | eee. 495.5 
Mm | @ } @ | Ce mec ede! | %1292.2 
mam i; 2 :} @ | GRY Cg Rs 5) Rn %Y98.5 
mee; 1 |} 6 ieee Yee ree | “Se@.5 : 
Meee: NIC: Number Incorrect Cheracters 
NzeC: Number Entries Corrected 
NICE: Number ct Incorreéct iBntries = NIC + NEC 
NMC: Number Missing Cheracters (same as NME) 
NCE: Nunber Correct entries 
INE: Total Number Errors 





Here i AL 


SUBJECT FRRORS: TARGET INFORMATION VERIFICATION TASK 


(DOUBLE-SPACEL IORMAT) 


— GED GD a= = a= Cem —— amp op amp = @= OP ot SD Sew SP ow ww Ss = OP oe © ow SD S@® 2 SD SP 2 Se SC Sw SC Sw SC CO CC OW SS SC SD CP SS SE 2! SC OD OS 2D CD Ow C C2 SE Se Oo ow 


fee, @¢ | 4 | feet erce “) 441 | %99.1 | 
me) 6fGlCU |lCGllCU! fees. | 166 | POs 1 
wee l6Uil C(t‘ SCC eeeCuerec | a1 =} %29.4 
mee’. UdhlUC<“ SCG CeComoumtargs | aoe | %96.2 | 
me ' ¢ | @ ! rn eee! toe %182.2 | 
<i a es Cee eee Sme | 14) | ZSE.3 
sa a oe (Coca (head. | 497.8 
oe, 6m6klC<“,:é‘i‘i ;#*‘*‘*! EO anole. toe | %96.3 
Ss a rr ae A a 3/2) es oa 292.8 
iC i re epi, oreo l<as4- | %96.3 
meee, fC |} Lt | etn lots 4 od) | 497 .O 
a See joe ees. ; Sa OC; pallor ; 
m }; i i! 3 $ Peete oe nh Se! %93.5 
3) rs Ale Bice tae | ZEE.3 
‘Sn es or Pee ees easy | | %94 .9 
15S rn cr Smee aie | ea | %E9.4 
ieee, @€¢& {| Ss | Sees Seles aaa | %27 2 
io) ec Se alee ie %93.4 
LC) i Ce ae eee Joe, | %94.9 : 
a! @! § ! Cae eo ees ise Gin | 


BPEGEND: NIC: Number [Incorrect Characters 
NitGeeNntmoer fntries Corrected 
WGN ben Oo: “lIncoOrrect Fntries = NIC + NEC 
NMC: Number Missing Characters (sare as NME) 
NGtoemvwember Correct Entries 
TNE: Total Number Errors 


SE 





TABLE XII 


SUBJECT ERRORS: TARGET INFORMATION VERIFICATION TASK 


(INVERSE-VIDEO FORMAT) 


—_— —— 2 eee Oe ee eee ee OP we ee 6 oe = OP = OF oe OF 6 OF = FF ow C= 6 Oe 6 6 6 6 OF 2 Oe SF © oP OF © Of 68 6 © ow OO! 6" 6! Os Sw 6 Sw 8 OP S62 8 C= Se 


ieee: Nic | Nec {| Nick ‘ Nwc | NCE ! TNE [/% CORRECT ENTR. 


ao ee ee a a ee ieee i ei) es eee ai ee ee ee es es es a is ee i eas ii es es i rs ee eee ee Se oe ee es Se oe 2 = Sw ew 2S Se 6S ew SB! SO SE SP SPS S_ 


Se (ee eee iis 44S 4126 .2 
2 ns er Go NEG ole ope ies 2 %95.@ : 
e668 UCU eee is ais %122 .@ 
A, | i in a” (eos 6 IS 27 5 
a Ss ae “eee ave | 415. I %93.9 | 
5) a i ee ee GA 7. 2) Ti oe ee oo SYE.3 | 
2 (i ie ee are ieee eis, | eG: CS %91.2 
ay Oo Oe COS 15h Ct %102.¢ | 
me, 2g ¢ y | CeCe te aie | YO7 .4 ; 
fo | @ | 2 eee ac Ot %95.Q 
meee ¢ "| 18 |; © |; 9" ;, 115 | LE4 25 
2 (ee Oe eps | %93.9 
mee) 6lUCUC tC Ve Oo age! | ey %89 .2 : 
12,” i ec emis 2) 127. %E6.7 
me ' h6U68lC IT CUCU (Ch Se eee |! 797.5 
Ne 2 ee ee ee 2 YEP 4 : 
me) 2 |} ei] PmeCee le Sls (| BIS 
i, eC | 6 } CU) are ee, oy eS: %182.@ 
1 i Seo cee fe poonS | 
0 i en it eS RES i ee oe %G9.1 


— ee ee ee ce es ee ee ee ee es es ee ce ee ee ce ee ee ee ee ee ee ee ee ee ce ee eee ee ee ee ee ee eee ec ee ee ee es ee ee ee ee 
—_—_ eee ee eee) ee eee ee ee ee ce es ee ce ce ee es ee ee i ee ee ce ee ee es es ee ee es ce ee es a 


meeendD: NiC: Nunber Incorrect Characters 
NEC: Number fntries Corrected 
NDGteeumber Of incorrect Entries = NIC + NEC 
NMC: Nurber Missing Characters (sare as NME) 
NCh:s Number Correct Entries 
i eelo tale Number Errors 


a 





ALT 


TABLE 


ENTRY 


ERRORS* CURING VOICE 


TOTAL 


TARGET SETS 


Sabie 


ee ce ect ccm mts es cm cs es ce ee ee es eee oe oe oe 


ec mm ce mc wey ee ts ee ges ee ee 


bel el eR en ee 


WOr~PH MM. VWOVOUNHQWIO AN WOWDOC YW) 

HAIN cied Nad ian! (VON 

WONONDA ADDIE DAMA O DEO VUVDSR 
AN dad e|4 | cA (Qc cf 


— ee cc ce cee ee ee ce ee eee ee ee ee ee ee ee ee ee ee 


QM wv) 
ie SS 


Ct WwW 
(YN co 
+2 OO 
Wo 
ec 


jSHommeccenit bons end Misreéccenitions 


he 





TABLE XIV 
NONRECOGNITICNS OF UTTERANCES BY ENTRY 


i; | G2 

il 49 

2 | 6 | 

S Sl ! 

4 iL ne 

IS iz 

6 44 

1 3 

g 4Q 

g 24 ! 
Gomerol , S 
meme y | 1y 
enter | EQ 
yes | 4 ! 
alpha | E 
bravo | 16 
charlie} ", ! 
lett | an, 
Rent | 3 


—— ce ee ee ee ce ee ee ee eee ee 
— oe oe ee es ee es ees ees es ee ee eee ee ee ee ee ee ee a 


=o 


ae 


igh. e- 


rere? 92 “2 


aogat dada TANK 


2 && @ =e @ Oo @o- => 6 @5- ae 
- 
7 


o- = 
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